Neural networks refer to a computational architecture modeled after biological brains. Within a neural network, nodes referred to as neurons may be interconnected and operate collectively to process complex input data. Examples of different types of neural networks include, but are not limited to, Convolutional Neural Networks, Recurrent Neural Networks, Deep Belief Networks, Restricted Boltzman Machines, etc. In a feed-forward neural network, the neurons of the neural network have links to other neurons. The links only extend in one direction, i.e., the forward direction, through the neural network.
A neural network may be used to extract “features” from complex input data. The neural network may include a plurality of layers. Each layer may receive input data and generate output data by processing the input data to the layer. The output data may be a feature map of the input data that the neural network generates by convolving an input image or a feature map with convolution kernels. Initial layers of a neural network, e.g., convolution layers, may be operative to extract low level features such as edges and/or gradients from an input such as an image. The initial layers of a neural network are also called feature extraction layers. Subsequent layers of the neural network, referred to as feature classification layers, may extract or detect progressively more complex features such as eyes, a nose, or the like. Feature classification layers are also referred to as “fully-connected layers.”
External memory can be used to store a large amount of intermediate results data generated during execution of the neural network. External memory can also be used to store a large number of weights used in the feature classification layers.